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Abstract — Consider the situation where a word is chosen 
probabilistically from a finite list. If an attacker knows the 
list and can inquire about each word in turn, then selecting 
the word via the uniform distribution maximizes the attacker's 
difficulty, its Guesswork, in identifying the chosen word. It is 
tempting to use this property in cryptanalysis of computationally 
secure ciphers by assuming coded words are drawn from a 
source's typical set and so, for all intents and purposes, uniformly 
distributed within it. By applying recent results on Guesswork, 
for i.i.d. sources it is this equipartition ansatz that we investigate 
here. In particular, we demonstrate that the expected Guesswork 
for a source conditioned to create words in the typical set grows, 
with word length, at a lower exponential rate than that of the 
uniform approximation, suggesting use of the approximation is 
ill-advised. 

I. Introduction 

Consider the problem of identifying the value of a discrete 
random variable by only asking questions of the sort: is its 
value X? That this is a time-consuming task is a cornerstone 
of computationally secure ciphers 0. It is tempting to appeal 
to the Asymptotic Equipartition Property (AEP) |2|, and the 
resulting assignment of code words only to elements of the 
typical set of the source, to justify restriction to consideration 
of a uniform source, e.g. Q, |4), J5). This assumed uniformity 
has many desirable properties, including maximum obfustica- 
tion and difficulty for the inquisitor, e.g. In typical set 
coding it is necessary to generate codes for words whose 
logarithmic probability is within a small distance of the word 
length times the specific Shannon entropy. As a result, while 
all these words have near-equal likelihood, the distribution is 
not precisely uniform. It is the consequence of this lack of per- 
fect uniformity that we investigate here by proving that results 
on Guesswork Q, 0, 0, (TO), fl~D extend to this setting. 
We establish that for source words originally constructed from 
an i.i.d. sequence of letters, as a function of word length it 
is exponentially easier to guess a word conditioned to be in 
the source's typical set in comparison to the corresponding 
equipartition approximation. This raises questions about the 
wisdom of appealing to the AEP to justify sole consideration 
of the uniform distributions for cryptanalysis and provides 
alternate results in their place. 

II. The typical set and Guesswork 

Let A = {0, . . . , m — l}bea finite alphabet and consider a 
stochastic sequence of words, {VFfc}, where Wk is a word of 



length k taking values in A k . The process {VKfc} has specific 
Shannon entropy 

H w := - lim j V P(W k = w) log P(W k = w), 



k 

and we shall take all logs to base e. For e > 0, the typical set 
of words of length k is 



we A k : er k{Hw+t) < P(W k = w) < e 



k(H w 



For most reasonable sources [2], P(Wk £ T|) > for all 
k sufficiently large and typical set encoding results in a new 
source of words of length k, WL with statistics 



( P(W k 



i P(W k £ Tl) 



if w £ Tg, 
ifwi Tt. 



(1) 



Appealing to the AEP, these distributions are often substituted 
for their more readily manipulated uniformly random counter- 
part, U%, 

P(UI=w):=\\n\ fc ' (2) 

[0 ifw$T£, 



where \T%\ is the number of elements in TL While the distri- 
bution of Wf. is near-uniform for large k, it is not perfectly 
uniform unless the original W k was uniformly distributed on 
a subset of A k . Is a word selected using the distribution of 
W| easier to guess than if it was selected uniformly, I7|? 

Given knowledge of A fe , the source statistics of words, say 
those of Wk, and an oracle against which a word can be tested 
one at a time, an attacker's optimal strategy is to generate a 
partial-order of the words from most likely to least likely and 
guess them in turn [12|, |7|. That is, the attacker generates a 
function G : A fe {1, . . . ,m k } such that G(w') < G(w) if 
P(Wk = w') > P(Wk = w). The integer G(w) is the number 
of guesses until word w is guessed, its Guesswork. 

For fixed k it is shown in [ 12 1 that the Shannon entropy of 
the underlying distribution bears little relation to the expected 
Guesswork, E(G(Wk)), the average number of guesses re- 
quired to guess a word chosen with distribution Wk using 
the optimal strategy. In a series of subsequent papers 0, (8), 
||9l , ifTOl , under ever less restrictive stochastic assumptions 
from words made up of i.i.d. letters to Markovian letters to 
sofic shifts, an asymptotic relationship as word length grows 



between scaled moments of the Guesswork and specific Renyi 
entropy was identified: 



Km -log E{G(W k ) a ) = aR w ( -J- 
fc->oo k V 1 + 



(3) 



for a > — 1, where Rw(f3) is the specific Renyi entropy for 
the process {M'fc} with parameter (3 > 0, 



1 1 



R W (P) := lim 

fc— s-oo fc 1 



/9 



log p ( w * = w y 



These results have recently ifTTI been built on to prove that 
logG(Wfc)} satisfies a Large Deviation Principle (LDP), 
e -g H 1 31 . Define the scaled Cumulant Generating Function 
(sCGF) of {k- 1 log G(Wfc)} by 

A w (a) := lim i \ogE (e alo&G{w "A for aeR 

fc— >oo K V / 

and make the following two assumptions. 

• Assumption 1: For a > — 1, the sCGF Aw(a) exists, is 
equal to aRw (1/(1 + <%)) an d nas a continuous deriva- 
tive in that range. 

• Assumption 2: The limit 



gw := lim \ \ogP{G{W k ) = 1) 
fc— >oo /e 



(4) 



exists in (— oo, 0]. 
Should assumptions 1 and 2 hold, Theorem 3 of IfTTI es- 
tablishes that Aw (a) = gw for all a < —1 and that 
the sequence {k~ l log G(Wk)} satisfies a LDP with a rate 
function given by the Legendre Fenchel transform of the 
sCGF, A w (x) := sup aeR {xa — Aw(ot)}. Assumption 1 
is motivated by equation ([3]), while the Assumption 2 is a 
regularity condition on the probability of the most likely word. 
With 



lim — Aw (a), 
c4-i da 



(5) 



where the order of the size of the set of maximum probability 
words of W k is exp(fc7^) IfTTI . A w (x) can be identified as 

-x — gw if x G [0, 7iy] 

sw a eM.i xa ~ A w {a)} if x G (j w , log(m)], (6) 
-oo if x £ [0, log(m)]. 

Corollary 5 of 1111 uses this LDP to prove a result suggested 
in OH, E), that 

1 



lim jE{\og{G{W k ))) = H w , 



(7) 



making clear that the specific Shannon entropy determines the 
expectation of the logarithm of the number of guesses to guess 
the word W k . The growth rate of the expected Guesswork 
is a distinct quantity whose scaling rules can be determined 
directly from the sCGF in equation ([3J, 

lim \\ogE{G(W k ))=A w {\). 

k— >oo k 



From these expressions and Jensen's inequality, it is clear that 
the growth rate of the expected Guesswork is less than Hw- 
Finally, as a corollary to the LDP, [ 1 1 1 provides the following 
approximation to the Guesswork distribution for large k: 

P{G(W k ) = n) w 1 exp (-fcA^fc- 1 logn)) (8) 



for n E { 1 , . 



. , m' 



'}. Thus to approximate the Guesswork 



distribution, it is sufficient to know the specific Renyi entropy 
of the source and the decay-rate of the likelihood of the 
sequence of most likely words. 

Here we show that if {Wfc} is constructed from i.i.d. 
letters, then both of the processes {U^} and {M^} also 
satisfy Assumptions 1 and 2 so that, with the appropriate 
rate functions, the approximation in equation ([8]) can be used 
with or in lieu of W k . This enables us to compare 
the Guesswork distribution for typical set encoded words with 
their assumed uniform counterpart. Even in the simple binary 
alphabet case we establish that, apart from edge cases, a word 
chosen via W| is exponential easier in k to guess on average 
than one chosen via U^. 

III. Statement of Main results 
Assume that the words {14^} are made of i.i.d. letters, 
defining p = (p , • • • ,Pm-i) by p a = P{W! = a). We 
shall employ the following short-hand: h(l) := — ^ l°g^a 
for I = (Z ,...,J m _i) G [0,1]™, l a > 0, J2J a = 1, 
so that H w = h(p), and D(l\\p) := - Y, a l °- lo g(Pa/D- 
Furthermore, define l~ G [0, l] m and 1+ G [0, l] m 

r G argmax{/i(Z) : h(l) + D(l\\p) - e = %)}, (9) 

1+ G argmax{/i(Z) : h(l) + D(l\\p) + e = h{p)}, (10) 

should they exist. For a > — 1, also define l w (a) and 77(a) 
by 



r)(a) 



Pa 



(1/(1+")) 



for all a G A and 



Ea S APa /(1+Q) l0gp a 



l/(l+«) 



(ID 



(12) 



Assume that h(p)+e < log(m). If this is not the case, log(m) 
should be substituted in place of h(l~) for the {U^} results. 
Proofs of the following are deferred to the Appendix. 
Lemma 1: Assumption 1 holds for {E/|} and with 



and 



where 



l*(a) 



A uc (a) := ah(l ) 



A w ,{a) = ah(l*(a)) - D(l*(a)\\p), 



1+ if 77(a) < -h{p) - e, 

l w (a) if 77(a) G {—h(p) — e, h{p) + e), (13) 

l~ if 77(a) > -h(p) + e. 



Lemma 2: Assumption 2 holds for {Uf.} and {W|} with 
gut = —h(l~) and 
g W e = min I -h(p) + e, log max p a 

Thus by direct evaluation of the sCGFs at a = 1, 
lim -logE(G(m)) = h(l~) and 

fc— yoo k 

lim ilog£7(G(W2))=A W r.(l). 

fc— >oo fc 

As the conditions of Theorem 3 ifTTl are satisfied 

lim i^(log(G(^)) = AJ,.(Q) = ft(r) and 
lim ^(log(G(^)) = A^«(0) = fc(p), 

fc— >oo fc 

and we have the approximations 

P(G(Ul) = n) « - cxp (-M^cffc- 1 logn)) and 
n 

P(G(^ e ) = n) « - cxp (-fcA^. (fc^ 1 log n)) . 
n 

IV. Example 

Consider a binary alphabet A = {0, 1} and words {Wfe} 
constructed of i.i.d. letters with P{W\ = 0) = po > 1/2. In 
this case there are unique l~ and l + satisfying equations (|9]l 
and ( [Tol l determined by: 

e 



l a = Po + 



log(po) - log(l - po)' 
e 



log(po) - log(l - po)' 

Selecting < e < (log(p )-log(l-p )) min(p -l/2, l-p ) 
ensures that the typical set is growing more slowly than 2 k and 
that 1/2 < Iq < po < l£ < 1. 

With l w (a) defined in equation (JTTJ, from equations <|3j 
and (HI) we have that 



if a < —1, 



A (a) = i log ^ ^ 

' }a/i(Z w (a))-.D(Z W (a)||p), if a > -1. 



log(po) 

(l + a)log^po^ + (l-po)^ 

From Lemmas [JJ and [2] we obtain 

—h(l~) if a < -1, 



if a < — 1, 
if a > -1, 



most likely words from smallest to largest is: unconditioned 
source, conditioned source and uniform approximation. 

From these sCGF equations, we can determine the average 
growth rates and estimates on the Guesswork distribution. In 
particular, we have that 

lim \E{\og{G{W k ))) = AV(0) - h(p), 

fc— >oo fc 

lim ^(log(G(W|))) = A^.(0) - %), 

k— >oo fc 

lim ^(log(G(Z7|))) = A'^0) = fc(r). 

A:— >oo fc 

As 1 — x)) is monotonically decreasing for x > 1/2 

and 1/2 < < po, the expectation of the logarithm of the 
Guesswork is growing faster for the uniform approximation 
than for either the unconditioned or conditioned word source. 
The growth rate of the expected Guesswork reveals more 
features. In particular, with A = rj(l) — (h(p) + e), 

lim hog E(G(W k )) = 21og(p| + (1-po)*), 

li m ilog^(GTO) = ( 21o ^o+(l-Po)^),A<0 

lim ilog£?(G(C^)) = Mr). 

fc— >oo fc 

For the growth rate of the expected Guesswork, from these 
it can be shown that there is no strict order between the 
unconditioned and uniform source, but there is a strict or- 
dering between the the uniform approximation and the true 
conditioned distribution, with the former being strictly larger. 

With e = 1/10 and for a range of p a , these formulae are 
illustrated in Figure [JJ The top line plots 

lim \E{\og{G{Ut)) - Iog(G(W fc ))) 

k— >oo fc 

= lim l -E{\og{G{Ut)) - log(G(lO) - HH - Hp), 

fc— foo fc 

showing that the expected growth rate in the logarithm of the 
Guesswork is always higher for the uniform approximation 
than both the conditioned and unconditioned sources. The 
second highest line plots the difference in growth rates of the 
expected Guesswork of the uniform approximation and the 
true conditioned source 

iim i io g wm 

k^L k 8 E{G{W£)) 



Ay. (a) 



ah(l~) ifa>-l, 



and 



A W e (a) = ah(l*(a))- D(l*(a) ||p), 



where I* (a) is deinfed in equation ( fT3| l and 77(a) defined in 
equation ( fl2] >. 

With 7 defined in equation we have 7^ = 0, jue = 
h(l~) and j w , = h{l+) so that, as h(l~) > h(l + ), the 
ordering of the growth rates with word length of the set of 



h(l~) -21og(p s 
D(l-\\p) 



Po) 5 ) if tj(1) < h(p) + e 
if 77(1) > hip) + e. 



That this difference is always positive, which can be estab- 
lished readily analytically, shows that the expected Guess- 
work of the true conditioned source is growing at a slower 
exponential rate than the uniform approximation. The second 
line and the lowest line, the growth rates of the uniform and 
unconditioned expected Guesswork 
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Fig. 1. Bernoulli(poi 1 ~ Po) source. Difference in exponential growth rates 
of Guesswork between uniform approximation, unconditioned and conditioned 
distribution with e = 0.1. Top curve is the difference in expected logarithms 
between the uniform approximation and both the conditioned and uncondi- 
tioned word sources. Bottom curve is the log-ratio of the expected Guesswork 
of the uniform and unconditioned word sources, with the latter harder to guess 
for large po ■ Middle curve is the log-ratio of the uniform and conditioned word 
sources, which initially follows the lower line, before separating and staying 
positive, showing that the conditioned source is always easier to guess than 
the typically used uniform approximation. 



initially agree. It can, depending on po and e, be either positive 
or negative. It is negative if the typical set is particularly small 
in comparison to the number of unconditioned words. 

For po = 8/10, the typical set is growing sufficiently 
quickly that a word selected from the uniform approximation 
is easier to guess than for unconditioned source. For this 
value, we illustrate the difference in Guesswork distributions 
between the unconditioned {W&}, conditioned {W^} and 
uniform {U^} word sources. If we used the approximation in 
|8]l directly, the graph would not be informative as the range 
of the unconditioned source is growing exponentially faster 
than the other two. Instead Figure [2] plots — x — A* (x) for 
each of the three processes. That is, using equation ([8]) and its 
equivalents for the other two processes, it plots 

y log G(w), where G(w) e {l,...,2 fc }, 
k 

against the large deviation approximations to 



1 



1 



1 



log P(W k =«>),- log P(W£ = w) and - logP(E7£ = 



k 



as the resulting plot is unchanging in k. The source of the dis- 
crepancy in expected Guesswork is apparent, with the uncon- 
ditioned source having substantially more words to cover (due 
to the log x-scale). Both it and the true conditioned sources 
having higher probability words that skew their Guesswork. 
The first plateau for the conditioned and uniform distributions 
correspond to those words with maximum highest probability 
(slowest exponential decay-rate). 



Fig. 2. Bernoulli(8/10, 2/10) source, e = 0.1. Guesswork distri- 
bution approximations. For large k, x-axis is x = l/fclogG(w>) for 
G(ui) £ {l,...,2 fe } and the j/-axis is the large deviation approximation 
l/k\o S P(X = w) « -as- A* x (x) for X = W k ,W£ and X = U%. 



V. Conclusion 

By establishing that the expected Guesswork of a source 
conditioned on the typical set is growing with a smaller expo- 
nent than its usual uniform approximation, we have demon- 
strated that appealing to the AFP for the latter is erroneous in 
cryptanalysis and instead provide a correct methodology for 
identifying the Guesswork growth rate. 

Appendix 

Note that by the definition of Tl as a typical set, P(W k G 
Tf) > 1 — e for all k sufficiently large and thus 

lim yio g p(w k en)=o, 

k— too K 

which we will use in the proofs of both lemmas. 

The proportion of the letter a € A in a word w — 

(u>i, . . . , Wf.) € A fc is given by 

||1 < i < k : Wi = ail 
n k (w,a) := n = = '- T X 

The number of words in a type /, where / € [0, 1] for all a € A 
and EaeA la = !> is S iven b Y 

N k (l) := \{w £ A k such that n k {w, a) = l a Va € A}|. 

The set of all types, those just in the typical set and smooth 
approximations to those in the typical set are denoted 

L k := {I : 3w e A k such that n k (w, a) = l a Va e A}, 
L% :— {I : 3w € T t:k such that n k (w, a) — l a Va G A}, 



U 



j/:^Z a logp Q £ [-h(p)-e,-h(p) + e]\j, 



where it can readily seen that L e k C L e for all k. 



For {C/|} we need the following Lemma. 
Lemma 3: The exponential growth rate of the size of the 
typical set is 



lim rlog|T fc e 

fc— yoo fc 



log to if log to < h(jp) + e 
h(l~) otherwise. 



where I is defined in equation Q. 

Proof: For fixed fc, by the union bound 

/,! < \m < (fc + l) m max: '' ! 



For the logarithmic limit, these two bounds coincide so con- 
sider the concave optimization problem 

fc! 

max : 



We can upper bound this optimization by replacing L\ with 
the smoother version, its superset L e . Using Stirling's bound 
we have that 



lim sup — log sup 



fc! 



fc 



um Jlog(m) if h(p) + e > log(m) 
< sup Ml) = < , N ' 

W \/i(r) if %)+e<log(m). 

For the lower bound, we need to construct a sequence 
such that /( fc ) ^ £| for all fc sufficiently large and h(l^) 
converges to either log(m) or h(l~), as appropriate. Let I* = 
(1/to, . . . , 1/to) or l~~ respectively, letting c € argmaxp a 
and define 



c, 



fc"HH*J 



if a 7^ c. 



Then j( fc ) € L| for all fc > -mlog(p c )/(2e) and fr(jW) 
/i(Z*), as required. 

Proof: Proof of Lemma [T] Considering first, 



a lim rlogl^l 

fc— yoo fc 



by Lemma|3] To evaluate Aw (a), as for any neN and a > 
Y^i a > J x a dx, 

again using Lemma [3] we have 

ah(l' 



= lim — log 

k-yoo fc 1 + 



< lim ylogE(e alosG( ~ u ^) 

fc— yoo fc 

1 1 Wl 

lim — log — — r > i a 



k— yoo fc 
1 



i=l 



where we have used Lemma [3] The reverse of these bounds 
holds for a E (—1,0], giving the result. 

We break the argument for {W 7 !} into three steps. Step 1 
is to show the equivalence of the existence of Aw (a) and 
aRw 0-/(1 + a )) for a > —1 with the existence of the 
following limit 



lim i log max { N k (l) 1+a T\ p, 



kl„ 



(14) 



Step 2 then establishes this limit and identifies it. Step 3 shows 
that A' w ,(a) is continuous for a > — 1. To achieve steps 1 and 
2, we adopt and adapt the method of types argument employed 
in the elongated web-version of |8]. 

Step 1 Two changes from the bounds of [ 8 1 Lemma 5.5 are 
necessary: the consideration of non-i.i.d. sources by restriction 
to T|; and the extension of the a range to include a £ (—1,0] 
from that for a > given in that document. Adjusted for 
conditioning on the typical set we get 



' max \N k {l) 1+a 



UaeAPa" 1 



1 + a i /.; [~ - w H w&n P(W k =w)f 
< E(e alosG ^) < (15) 



(fc + l) m ( 1+Q )majJ N k (lf 



YlaeAPa" 1 



■>weT: 



P(W k = 



The necessary modification of these inequalities for a 6 
(-1,0] gives 



l+q I\a<£APa la 

< E(e al ° eG(w * y ) < 



max < N k (l) 



(16) 



max^ iVfc(t) 



UaeAPa 1 " 



1 + a iiLj [' vv P(W fc = w) j ' 

To show the lower bound holds if a € ( — 1 , 0] let 

Z G argmax ^ JV fc (0 = 1— r V . 

Taking lim inf k ^oo fc -1 log and limsup^^ fc -1 log of equa- 
tions (jT3J and ( ff6l > establishes that if the limit ( fl4"| > exists, 
Avk e (a) exists and equals it. Similar inequalities provide the 
same result for aRwO/0 + a)). 

Step 2 The problem has been reduced to establishing the 
existence of 



lim — log max < N k (l) 

fc— yoo k l€Li 



l+a 



n 



Pi 1 " 



a€i 



< lim -log|T fc r = ah(l~), 

k— yoo fc 



and identifying it. The method of proof is similar to that 
employed in Lemma [T] we provide an upper bound for the 
limsup and then establish a corresponding lower bound. 

If /( fe ) ->■ I with G L k , then using Stirling's bounds we 
have that 

lim hogN k (lM) = h(l). 

k— yoo fc 



This convergence occurs uniformly in I and so, as L e k c L e 
for all k, 



lim sup — log max < Nk (I) 

k— >oo k '£^1 



l+a 



< sup f (1 + a)h{l) + V l a \og Pa J 
V a J 

= sup (ah(l) - D(l\\p)) , (17) 

This is a concave optimization problem in I with convex 
constraints. Not requiring I £ L e , the unconstrained optimizer 
over all I is attained at I (a) defined in equation ( fTT) , which 
determines 77(a) in equation (12) . Thus the optimizer of the 
constrained problem ( fl7] > can be identified as that given in 
equation ( fT3) . Thus we have that 

lim sup I log max <j 7V fe (/) 1+Q JJ p™° \ 



k— >oo 



<oft(r(a)) + D(r(a)||p), 



where Z*(a) is defined in equation ([13). 

We complete the proof by generating a matching lower 
bound. To do so, for given I* (a) we need only create a 
sequence such that -» Z*(a) and £ L% for all k. 
If /*(a) = Z~, then the sequence used in the proof of Lemma 
[3] suffices. For Z* (a) = l+, we use the same sequence but with 
floors in lieu of ceilings and the surplus probability distributed 
to a least likely letter instead of a most likely letter. For 
l*{a) — l w (a), either of these sequences can be used. 

Step 3 As Aw(a) = ah(l*(a)) - D(l*(a)\\p), with I* (a) 
defined in equation ( fl3") , 

■^-h w ,(a) = h(l*(a)) + kw>{a)-^-l*{a). 
da da 

Thus to establish continuity it suffices to establish continuity of 
I* (a) and its derivative, which can be done readily by calculus. 

■ 

Proof: Proof of Lemma [2] First consider 

gjje — lim — max log P([/| = w) 

fc->oo k WET? 



lim — log - — 
fe->oo k \Ti. 



-Kn> 



using Lemma |3] 

For {W^}, if gw < ~h(p) +e the result follows simply, so 
assume that this is not the case. By the property mentioned at 
the beginning of this section, the normalisation doesn't play a 
role in the limit, i.e. 

lim sup — log max P(Wt — w) 
= lim sup — log max P(Wk = w) 

fe^oo k toST« 

with an analogous equality for the lower bound. As P{Wk = 
w) < exp(—k(h(p) — e)) for all w £ Tj:, the upper bound 



follows immediately and we need the corresponding lower 
bound on 

liminf — log max P(Wk = w). 

k^co k w <= T k 

If gw > — h(p) + e, there exists K £ N such that for all 

k > K, fc _1 logmax^gAt P(Wk = w) > —h(p) + e. Then 
taking k > K, 

nnur \\ -k(h(n)+e) mm aSAPa 

max PlWk — w) > e V J . 

w&T' k max beA Pb 

To prove this, we use proof by contradiction. Assume 

T>t\xr \ ^ -k(Mp)-e) mm aeAPa 

max P( Wk = iv) < e v yF ' ' . 

w&T' k max beA P6 

Take a word w* £ arg max^g^ P(Wk = w), there exists 
at least one letter in w* , b £ A, such that p^ < max aG Ap a 
as P{Wk — w*) < ma,x weA k P{Wk — w). We then replace 
one occurrence of b in w* with an element of argmax aSj 4p a 
to make the k letter word w'. Then fc _1 logP(Wk = w*) < 
k~ 1 \ogP(Wk — w'). As for each k we have only changed 
one letter and by assumption, 

^\ogP{W k =w') < 

1 i (vnzr ^max 6eA p 6 \ 

- log P{W k = w )— < -h(p) + e. 

k \ min oeA p a J 

This implies w' £ Tt and contravenes our choice of w* . So 
liminf k^ 1 log max P(Wk = w) > —h(jp) + e 

k— ¥00 wET k 

if gw > —h(p) + e. Lastly if g w = —h{p) + e, 
max P(Wk = w) > 

WET? 



'max w( z A k P(W k = w) 

if max weA k P{W k =w)< e - fe ( h (p)- e ) 
_ Hh{p) _ e) mmoEAPa otherwise ^ 
max heA p & 

The result follows as 

— h(p) + e + lim inf [ — log ™ nagA ^ a 
k-^00 \k maxhgAPh 

= lim inf — log max P(Wk = w) = —h{p) + e. 

fc->oo k w£A k 



Acknowledgment 

M.C. and K.D. supported by the Science Foundation Ire- 
land Grant No. 1 1/PI/l 177 and the Irish Higher Educational 
Authority (HEA) PRTLI Network Mathematics Grant. F.d.P.C. 
and M.M. sponsored by the Department of Defense under Air 
Force Contract FA8721-05-C-0002. Opinions, interpretations, 
recommendations, and conclusions are those of the authors and 
are not necessarily endorsed by the United States Government. 
Specifically, this work was supported by Information Systems 
of ASD(R&E). 



References 



[1] A. Menezes, S. Vanstone, and P. V. Oorschot, Handbook of Applied 

Cryptography. CRC Press, Inc., 1996. 
[2] T. M. Cover and J. A. Thomas, Elements of Information Theory. John 

Wiley & Sons, 1991. 
[3] J. Pliam, "On the incomparability of entropy and marginal guesswork 

in brute-force attacks," in INDOCRYPT, 2000, pp. 67-79. 
[4] S. Draper, A. Khisti, E. Martinian, A. Vetro, and J. Yedidia, "Secure 

storage of fingerprint biometrics using Slepian-Wolf codes," in ITA 

Workshop, 2007. 

[5] Y. Sutcu, S. Rane, J. Yedidia, S. Draper, and A. Vetro, "Feature 
extraction for a Slepian-Wolf biometric system using LDPC codes," in 
ISIT, 2008. 

[6] F. du Pin Calmon, M. Medard, L. Zegler, J. Barros, M. Christiansen, 
and K. Duffy, "Lists that are smaller than their parts: A coding approach 
to tunable secrecy," in Proc. 50 th Allerton Conference, 2012. 

[7] E. Arikan, "An inequality on guessing and its application to sequential 
decoding," IEEE Trans, Inf. Theory, vol. 42, no. 1, pp. 99-105, 1996. 

[8] D. Malone and W. Sullivan, "Guesswork and entropy," IEEE Trans. 
Inf. Theory, vol. 50, no. 4, pp. 525-526, 2004, http://www.maths.tcd.ie/ 
~dwmalone/p/guess02.pdf 

[9] C.-E. Pfister and W. Sullivan, "Renyi entropy, guesswork moments and 
large deviations," IEEE Trans. Inf. Theory, no. 11, pp. 2794-00, 2004. 
[10] M. K. Hanawal and R. Sundaresan, "Guessing revisited: A large devi- 
ations approach," IEEE Trans. Inf. Theory, vol. 57, no. 1, pp. 70-78, 
2011. 

[11] M. M. Christiansen and K. R. Duffy, "Guesswork, large deviations and 
Shannon entropy," IEEE Trans. Inf. Theory, vol. 59, no. 2, pp. 796-802, 
2013. 

[12] J. L. Massey, "Guessing and entropy," IEEE Int. Symo. Inf Theory, pp. 
204-204, 1994. 

[13] A. Dembo and O. Zeitouni, Large Deviations Techniques and Applica- 
tions. Springer- Verlag, 1998. 

[14] E. Arikan and N. Merhav, "Guessing subject to distortion," IEEE Trans. 
Inf. Theory, vol. 44, pp. 1041-1056, 1998. 

[15] R. Sundaresan, "Guessing based on length functions," in Proc. 2007 
International Symp. on Inf. Th., 2007. 



